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A MULTITRAIT-MULTIMETHOD MODEL FOR STUDYING GROWTH 



Abstract 

The logical structure of the Campbell and Fiske (1959) multi trait- 
multimethod approach is applied to the problem of studying growth and its 
determinants. The resulting model is a special case of Joreskog s ( 1970a) 
general model for the analysis of covariance structures. The relationships 
of traditional psychometric formulations to this model are detailed. 
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A MULT ITRAIT-MULT IMETHOD MODEL FOR STUDYING GROWTH 1 
Charles E. Werts, Karl G. j8reskog, and Robert L. Linn 

Werts and Linn (1970a) have suggested that a multitrait-multimethod 
approach (Campbell & Fiske, 1959) might, be used for studying growth. The 
purpose of this paper is to detail such a model and to outline implications 
for the study of growth. The major focus of our exposition will be the 
logic of this model rather than the estimation of parameters or testing the 
fit of the model to data. A comprehensive discussion of appropriate estima- 
tion and fit-testing procedures may be found in j8reskog ( 1970a), whose 
general model for the analysis of covariance structures subsumes the models 
used in this paper. 

i 

The Model 

The multitrait -multimethod approach may be treated as a problem in 
confirmatory factor analysis (j8reskog, 1970a., 1971)* For illustrative 
purposes we will consider the example of three traits and three methods 
since this is the minimum number of traits and methods required to produce 
unique (defined in J8reskog, 1969, pp. 185-186) parameter estimates, given 
the assumption that each observed measure loads on only one trait and one 
method factor and all factors are oblique. The general factor analytic 
model is: 

y = p + AT + e (l) 



T?he research reported herein was performed pursuant to Grant No. 
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where y is the vector of observed scores, 
l_i is the mean vector of y , 

A is a matrix of factor loadings, 

T is a vector of common factor scores, and 

e is a vector of unique factor scores corresponding to specific 

factors and/or errors of measurement. 

For our example : 

y* = ( y ll ,y 21 ,y 31 ,y 12 ,y 22 ,y 32 ,y l3 ,y 23 ,y 33^ ’ 



(la) 



where in y. . , i = method and j = trait, 
*^t3 



T' = (T 1 ,T 2 ,T 3 ,M 1 ,M 2 ,M 5 ) 



(lb) 



where T. = the j -th trait factor, 

J 

M. = the i -th method factor, 
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where A. . are loadings on trait factors and 
ID 

B. . are loadings on method factors. 

3-D 

The expected variance-covariance matrix Z of y is then given by 

Z = A M' + 0 2 (2) 

2 

where 9 is a diagonal matrix whose elements are the variances of e . 
Since all factors are oblique, in our example: 
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where the C 1 s are covariances and the V 's are variances. 

Following Jftreskog ( 1970a), parameters will be labelled as one of three 
kinds: (l) fixed parameters that have been assigned given values; (2) con- 

strained parameters that are unknown but equal to one or more other parameters ; 
and (3) free parameters that are unknown and not constrained to be equal to 
any othtr parameter. The term "identifiable" will be used in the sense defined 
by Fisher (1966, p. 25): "we shall speak of that equation as identifiable (or 

identified) if there exists some combination of prior and posterior information 
which will enable us to distinguish its parameters from those of any other 
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equation in the same form." For the models studied in this paper, the term 
"identifiable" is synonymous with the factor analyst's term "unique solution," 
i.e. , a solution is "unique" if all linear transformations of the factors 
that leave the fixed parameters unchanged also leave the free parameters 
unchanged. As JiJreskog (1970b) notes: "Before an attempt is made to esti- 

mate a model of this kind, the identification problem must be examined." The 
number of overidentifying restrictions on the model is frequently of interest, 
for example, after standardizing factor variances (i.e., = Vw = l) the 

j i 

three method by three trait model has three overidentifying restrictions, 
i.e., Z has 45 distinct variances and covariances as compared to 42 free 
parameters to be estimated (l8 factor loadings, 15 factor covariances in $ , 
and nine residual variances in 0 ). The number of overidentifying restric- 
tions is the degrees of freedom (df ) for the test statistic in J&reskog's 
general model (1970a, p. 24l, sec. 1.4). The "path analysis" approach used 
by Werts and Linn (1970a) can be very useful in exploring the identification 
question in overidentified models. However, as noted by Hauser and Goldberger 
(1970) the "path analysis" literature does not adequately deal with the esti- 
mation problem in overidentified models, in part because the sample-population 
distinction is blurred. 

The multitrait-multimethod approach considered above does not consider 
any functional relationships among the trait factors, i.e., the approach deals 
only with errors of measurement. In the study of growth, these trait factors 
correspond to initial status, final status, and the determinants of growth and 
a structural model showing the relationship among these variables must b 0 
specified. Substantive inferences about growth are based on estimates of the 
parameters of the structural model. 
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Suppose that the structural model for growth took the form: 

T 5 = D X T X + D 2 T 2 + 6 (5) 

where is the final status, T 2 is the initial status, and is a 

determinant of growth; all other influences on growth (represented by | ) 
being independent of and T 2 . In this model the initial status T 2 

may influence the rate of growth. The parameters of equation (5) are just 
identifiable in terms of the elements of $ , i.e., the number of restric- 
tions on the overall model is not changed. Assuming that and T 2 are 

measurements on the same dimension as implied by the terms "initial" and 
"final" status, growth (a) is equal to T^ - T 2 . Werts and Linn ( 1970b) 
have shown that the regression weights for and T 2 are: 







D 1 D AT 1 .T 2 ; 




GO 


and 




D 2 = 1 + 




(5) 


where 


D «P r T 2 1S the 


regression weight of A on 


T. with T n 
1 d 


controlled 


and 


D fflT 2 .T 1 is the 


regression weight of A on 


T 2 with T, 


controlled. 



In other words represents the direct influence of on growth and 

D 2 represents the direct influence of initial status on growth plus unity 
(which represents that part of which is initial status). Since = 

A + T 2 , substituting equations (^) and (5) into (5) yields: 



A “ D Ar x .T 2 T i + d at 2 .t 1 T 2 + 
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In terms of 1 , and £ , equations (lb), (lc), and (2a) become: 



T* = (T^T^M^M^Mj) 



(7a) 



and 
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respectively. If the analyst wished to scale a factor by the unit of a 
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particular measure this may be accomplished by setting the A. . slope for 

1 J 

the measure equal to unity (in which case the variance of the corresponding 
factor should not be standardized but left free to be estimated by the program). 
The assumption that Tg and are measures on the same dimension is 

equivalent to setting the same method regression weights equal, i.e., in our 
example A^g = A^ , Agg = Ag^ , and A^g = A ^ . As detailed by Werts and 
Linn (1970a) the effect of these restrictions is that the ratio of the vari- 
ance of to Tg is fixed. For estimation purposes it is convenient to 

standardize all factors except whose variance is fixed in relation to 

Tg . The model defined by equations (7a), (7h), and (7c) is no longer a 
simple factor analysis model, but may be estimated using Jtfreskog's ( 1970a) 
general model for the analysis of covariance structures. For this purpose 
A* may be rewritten as the product of two matrices: 



A* = BA** 
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and 



A** = 



A 
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and X l5 = B l5 /A l5 , = B 25 /A 25 , X^ = B^/A^ . By substitution: 



Z = bA**<&*A** * B * + e 2 , 

which is a special case of Jtfreskog's (1970a) general model. 

In using the computer program (j8reskog, Gruvaeus, & van Thillo, 1970) 
the parameters A^ 2 , A 22 , A,g in A** should be constrained to be equal 
to A .|3 , A 2 ^ , and A^ respectively in B . The resulting model has k^> 
distinct variances and covariances in 2 and ^0 free and constrained param- 
eters (17 in A** , l4 in , 9 in 0 , none in B because .of equality 

restraints), which means that the model has five overidentifying restrictions 
( df )„ The advantage of casting the analysis in terms of j8reskog’s general 
model is that, given the assumption that the observed variables are distributed 
normally, various hypotheses about the model may be tested in large samples. 
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In particular , we may wonder if trait factors are uncorrelated with methods 

factors and methods factors with each other as assumed by Cronbach and 

Furby (1970) and Werts and Linn ( 1970 a) in their analysis of growth. To 

make this test, the analysis would be run with the model of (la), (lb), and 

(lc), and ( 2 a) with V = V = V = V M = V M = V =1 and then the 

1 2 i 3 M 1 m 2 m 3 

analysis would be made with C m .. = C m .. = C m .. = C m .. = C_ .. = C m .. = 

T 1 M 1 ^ 1^2 T 1 M 3 T 2 M 1 T 2 M 2 T 2 M 3 



= C„ 



= C„ 



= C, 



= c. 



'T 5 M x "T 5 M 2 " T 3 M 3 SlgM^ “ 0 • For OUr example ; the 

initial analysis would yield a chi-square with three df for testing the fit 
of the model to the data. The second analysis would yield a chi-square with 
15 df since 12 additional restrictions have been made. The increase in chi- 
square with 12 df is a test of the tenability of the additional restrictions. 
Starting with the same initial model, the tenability of assuming that A^g = 

A 13 ; A 22 = A 23 3 and A 32 = A 33 W te tested (dropping the V T =1 

assumption) using the increase in chi-square with ■"2 df. Likewise starting 

with these assumptions (i.e., equations Cf’d), (7h), and (7c), and df = 5) 

’ 

hypotheses about growth can be tested, e.g., can be set equal to zero 

2 

and the resulting change in,. X (df = 1 ) is a test of whether T^ directly 

influences growth. To test whether initial status directly influences growth 

(i.e., whether D A ,_ „ = 0 ) , D 0 would be set equal to unity (see equation 

Ai 2* i l 2 2 

( 4 )), the increase in X (df = l) testing this hypothesis. The fit of 

the observed variance-covariance matrix S to the estimated elements of Z 

may be used to form some judgment as to changes in fit resulting from addi- 

2 

tional restrictions , especially when the X test is inappropriate because 
"the assumption of multivariate normality is- not reasonable. 
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As originally conceived by Campbell and Fiske (1959) "the multitrait- 
multimethod approach required each trait to be measured with each method, 
as in the example analyzed above. The linear structural model approach 
proposed herein requires that model parameters be identifiable, a question 
which is unrelated to whether each trait is measured with each method. -In 

order to fix the ratio of the variance of the final status to the initial 

status factor, only one pair of initial and final measures with the same 
units of measurement are required, i.e., the three sets of initial-final 
measures in our example serve to overidentify this variance ratio. The 
identification problem would be greatly simplified if one of these same 
method sets were replaced with different method measures, even though the 
resulting matrix would no longer be in the form required by Campbell and 
Fiske. Campbell and Fiske ‘s argument that different method measures of a 
trait are required to improve convergent validity appears fundamentally sound 
and is a basic premise in our analysis. We have abandoned the particular 
type of analysis used by Campbell and Fiske because it fails to specify the 
tmderlying structure being postulated, and does not allow for nonsymmetrical 
method-by-trait combinations . 

Relationship to Classical Test Theory 

The multitrait -multimethod formulation can be shown to include various 
procedures derived from classical test theory as special cases, e.g., the 
commonly used formulas for reliability of differences, correlation of true 
initial status with true gain, and the correlation of true scores over time 
can be derived from the multitrait -mult imethod model by imposing specifiable 
restrictions. To illustrate this point we shall examine the case of two 
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parallel measures (y ^ ; y 22 ) S iven initially and two finally (y-^ ; y 25 ^ 

First let us consider the analysis given the traditional assumptions that all 
errors of measurement are independent of each other and of the true scores. 

In our formulation this__is ...epAiivalest to asserting that there are no methods 



factors. Without further assumptions the model may be represented in terms 
of equation (l) as 



y (y^p ? ypp ? y q^ ? ^ * 

T = (T 2 ,T 3 ) , 
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(8e ) 



O 

ERIC 



13 



- 12 - 



Assuming that initial and final status are on the same scale, "'parallel" test 

assumptions are equivalent to (J&reskog, 1971) fixing = A ^ - A^ = A ^ = 1 

and constraining V = V and V = V . All parameters are identi- 

6 12 e 22 e l3 e 23 

fiahle and df = 5 • Identification still occurs without the error variance 
assumptions (df = 3) , i.e., in true score lexicon, "essentially tau-equiva- 
lent" measures (Lord & Novick, 1968, pp. 47-50) would suffice. If we choose 
to use nonparallel or "congeneric" (j&reskog, 1971) measures, one pair of 



measures over time being on the same scale (e.g., A 12 = A ^) , V T could 
be arbitrarily standardized (= l) , yielding an identifiable model with 



df = 1 . In all these cases, growth statistics may be obtained from the 
parameter estimates or the model can be transformed to obtain growth statis- 
tics directly. Inserting T, = T„ + A then: 
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where V_ =1 for convenience. 
i 2 
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Relevant growth statistics are: 



II 


correlation of initial status with gain = C™ A /v A V. , 
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Similarly if parameter estimates were derived from the original model of 
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Following jBreskog (1971) the parallel test assumption can be tested (given 
multivariate normality) by comparing the chi-square for the "essentially tau- 
equivalent" model to that for the "parallel" test model; the difference in 
chi-square with df = 2 is a test of assumptions that V = V and 

p p 

e 12 e 22 

V = V . Similarly the increase in chi-square from the "congeneric" 

13 S 23 

model to the "essentially tau- equivalent" model (df = 2) is a test of the 
assumptions that A^ 2 = A 22 and A^, = A 2 ^ . If the parallel test assump- 
tions are accepted then the population reliability at the initial time may be 
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estimated by V f (v + V ) and reliability at the final time by 
2 x 2 e l2 

V T t (V^ + ) . The reliability for each test is the square of the 



'3 3 "13 

corresponding standardized factor loading in the case of "essentially tau- 
equivalent" or "congeneric" measures. Another statistic of interest in the 
traditional psychometric literature is the reliability of differences (p^) 
which is defined as the true variance of the differences divided by the 
variance of the observed differences. In the parallel case the estimated 
population error variances can be used to obtain directly: 



V, 



- /N 



V. + V + V 
A e l2 e 15 



(12a) 



With "essentially tau- equivalent" assumptions no statement is made about 
equality of error variances so that four reliabilities may be estimated: 
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Formulas (12a), (12b), (12c), (l2d), and (l2e) are based on the assumption 
that the true scores have the same units as the observed scores, which is not 
true in the case of congeneric measures. Since the regression of observed on 
true differences is equal to the regression of observed on true scores (Werts & 
Linn, 1970a, equation (25)). it is only necessary to standardize this weight 
with the appropriate variances to obtain the reliability of differences for 



all cases, e.g., in the congeneric case if A^ 2 = A.^ then 



As As 2 

P A = A 12 



V, 



V 



+ v - 2C (yi2> *15) 



(I2f) 



y 12 y i3 



where V 



y-, 



12 



a A / \ ^ 
V and C(y, Q , y..,) are the estimated elements in Z 



m A 

This formula uses estimated elements in Z which are provided in the com- 
puter output for J8reskog's program (J8reskog, Gruvaeus, & van Thillo, 1970). 
The program computes the elements in Z from the estimates for the underly- 



ing parameters, e.g., c(y^g, y^, ) = ^ . This model (all measure- 

ment errors independent) may be used to clarify traditional procedures for 
obtaining growth statistics. For example, consider the case in which one 
initial and one final test is given. A common procedure is to obtain split 
half reliabilities at each time and use these to correct for attenuation. If 
y 12 and y 22 are the initial split halves and y_ and y 2 ^ the final 
split halves, this case corresponds exactly to the parallel measure case 
analyzed above. The difference from the traditional procedure is that the 
complete variance -covariance matrix for the split halves is computed and used 
in the analysis. As shown above, the "parallel" and "essentially tau-equiva- 
lent" assumptions can be tested against the congeneric model and the congeneric 
model is overidentified. From this perspective the traditional procedure 
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neglects useful information about correlations among split halves and thereby 
loses the possibility of rejecting the model because of poor fit to the data 
and of analyzing the data making only congeneric test assumptions. To under- 
stand the connection with the traditional formula it is of interest to 
standardize E into a correlation matrix (correlations generated by the 
model are indicated by symbol p ) and to show the relationships to standard- 
ized model parameters (denoted by asterisk): 



p(y 12 , 
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(13c) 
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(I5f) 



/S /\ A A 

If parallel test assumptions are valid then A* 2 = A g 2 and = A^ > In 

which case equations (l3a), (13b), (13c), and (l5d) are identical and should 
be recognized as the traditional correction for attenuation, except that the 
correlations are drawn from E rather than from the observed correlation 
matrix S . Equations (l3e) and (l3f), under parallel test assumptions, are 
simply the assumption that the reliability defined as the squared correlation 
(i.e., A*^ or A*^) observed with the true score is equal to the 

correlation between two parallel tests, but again the correlations are drawn 
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from E not from S . What these equations show is that it is not necessary 

for the reliabilities of the split halves to be equal in order to identify 

the unattenuated correlation p given uncorrelated errors. If the esti- 

i 2 i 3 

mates of the elements in E for the parallel case are examined it will be 
found that because of the structural specifications : V = V , V = V , 

j 12 y 22 y 13 y 23 

^ y l2 ; y i3 ) — ^23 ^ = < "'^ y 22 ; y i3^ = ® ^ 2 . 2 .’ y 23 ^ ; ^ y 12 ; y 22^ = } 

C(y 15 , y 2 j) = V T and C(y 12 , y^) = C(y 22 , = C T T . Translating the 

3 2 3 ^ 

equation for the reliability of differences into the elements of E : 



or 



P A = 



C(y 12 , y 22 ) + C(y 15 , y 25 ) - 2c?(y 12 , y 15 ) 

V r + T - 2c(y 12 , y l5 ) 



12 



13 



P A = 



^ y i2^ yi2 ' y 2 g ^ + y y 15 ^ y i3 J ’ y 23 } " 2p ( y i2’ 



12 y i3 



V 

y 



12 



v y - 2p(y 12 , y ) If V 
y l3 V y l2 y l3 



(l^a) 



. (14b) 



Equation (l^b) should be recognized as the traditional formula for the relia- 

A 

bility of differences, noting however that the estimates are drawn from E , 
not from the observed matrix S . The essentially tau- equivalent case differs 
from the parallel case in that the corresponding variances in E are not 
required to be equal, however the covariances between independent measures of 
different traits are still equal to the covariances between the corresponding 
traits factors. This means that formula (l^J-a) could be used for any pair of 
tau- equivalent tests over time. For congeneric measures the formula involves 
the pairs of measures which have the same units over time, e.g. , if A^ 2 = A^, 
then equation (l2f ) may be translated into 
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P A 



V y 12 (A 12 ) + < A ! 3 > - 



15 . (14c) 



v - +T - - 2p(y 12’ y 13> J%Jy a 



y 



12 



y 



15 



Equation (l4c) is the reliability of differences formula given by Werts and 
Linn (1970&> equation (26)) for the case of correlated errors over time for the 
pair of measurements on the same scale, i.e., the Werts and Linn formula is 
also appropriate to the independent error case when applied to the elements 
of £ rather than S . If formula (l4c) applies to correlated errors using 
congeneric measures then it may be specialized for the parallel measures case , 
e.g., if y^, and y^, have nonindependent errors and y^ 2 and- y 2 j 
independent errors : 



A A 



(a) A*^ = , by parallel test assumptions, therefore A^gA^p^ ^ = 

^12^5 Pf^ » 



A, A A 



(b} but p (y 12 , y 25 ) = A t7h\ h . 



Since 



A* 2 = /ft7 12 , y 22 ) , % =yp(y 13 » y 2 5 } ' 

equation (l4c) becomes 



= V 
y l2 y 22 



y 



15 



V , 

y 23 



A A 



V y 12 p(y 12' y 22> + V y 15 p(y 15’ y 2 3 > ' 2p(y 12’ ^ 



A A 



A 



V, + \j - 2p(y i2' y i5 



. (l4d) 



15 



Equation (l4d) is the formula for the reliability of differences for "linked" 
(i.e., correlated errors) parallel test measures given by Cronbach and Furby 
(1970, equation (6)), which can be seen to be the parallel measure specializa- 
tion of the Werts -Linn equation for nonindependent congeneric measures. 
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Similarly from equations (lla), (lib), (lie), (lid), and (lie) it follows 
that the estimated correlation of status with gain is : 



C - V 

T 3 T 2 T 2 



t 2 a 



A /A A _ A \ 

fy (v + v - 2c ) 

1 2 1 2 i 2 i 3 



(15a) 



In the congeneric case with A^ 2 = , this may be transformed into 



£ 



Vj 3 



” A 12 / y l2 



t 2 a 



P ^ 

A i2 v y 



12 



A O /\ 

A*Jv 

13 y 



- 2p, 



A A 



13 



A* A* 

'T C T, 12*13 
2 3 



V V 
y 12 y l3 



(15b) 



Formula (l5h) is the correlation of status with gain given by Werts and Linn 
(1970a, equation (28)) for the case of congeneric measures and correlated 
errors, i.e., the formula applies also to the independent error case. In 
the case of parallel independent measures ^ = p(y^ 2 , y.^) * 

/(S(y 1 2 , y 22^ y l3> y 23^ which when substituted into formula (15b) yields 
the traditional formula for the correlation of status with gain as applied 

A 

to the elements of Z : 



p ( y 12’ y 13 ) y V y - P < y 12' y 22)/ V y 



12 



t 2 a 



yp(y 12 , y 22 f^i2’ y 22 ) ^y l2 + p(y i 3 j y 23^ " - 2 p(y 12 , , 15 )/0 



13 



Our purpose in demonstrating relationships to traditional formulations is 
purely heuristic, since jBreskog's program yields estimates of model param- 
eters given the structural assumptions specified by the investigator, i.e., 
the traditional formulas apply to the elements of Z which are not directly 
observable but which are estimated as a function of the parameter estimates. 




. (15c) 
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TraditionaJ. psychometric approaches have dealt with models which are just 
identified which means that models which exactly reproduce the observed 
variance-covariance matrix can be employed (i.e., S = S )• The limitation 
in this approach is that overidentification is necessary if the fit of the 
model to the data is to be tested. 

In this paragraph we propose to use our model to specify the conditions 
implicit in Cronbach's (1960, pp. I56-I39) discussion of coefficients of 
"stability" and "equivalence." Cronbach uses an example in which two forms 
of the Mechanical Reasoning Test of the DAT were used, the same forms being 
used for test and retest purposes. When the same form is repeated, the test- 
retest correlation is higher than the test-retest correlation between differ- 
ent forms, suggesting the presence of "long-lasting test-specific" factors. 
The implication is that the errors of measurement for the same test repeated 
are not independent. Assuming that both forms were repeated and errors of 
measurement independent for different forms, the model for parallel measures 
is of the form: 

y = p + AT , (l 6 a) 

where 

y = (y 12 ; y 2 2> y \y y 25 > ( l6b ) 

where and y^ are "the same test as are y^ and y2^ • 

T = ^” 5 ^ e 12 ’ e 22 ; e i 3 > e 23 ^ 9 (l 6 c) 
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and 



A = 



0> = 



V, 



T , 



T 2 T 3 



1 

1 

0 

0 



V, 



T, 



0 



0 



0 



0 



0 

0 

1 

1 



V, 



E 



1 

0 

0 

0 



12 



0 



e 12 e 13 



0 

1 

0 

0 



0 

0 

1 

0 



0 

0 

0 

1 



Symmetric 



V_ 



'22 



0 



e 22 e 23 



V 

e 13 

o V_ 



'23 



(l6d) 



(l6e) 



where V 



= V 



V 



= V 



12 22 13 c 23 

The model of (l6a) is the special case of factor analysis in which the residual 
factors are treated as latent factors. Examination of $ shows that the same 



test errors of measurement are nonindependent, i.e., C and C 

e l2 e 22 



e 22 e 23 



/ 0 



All parameters are identifiable and df = 3 (10 distinct elements in 2 less 

7 free and constrained parameters). Essentially tau-equivalent assumptions 

would still have provided identification but with only one overidentifying 

restriction (since V / V , V f V ) . An interesting case occurs 

e 12 e 22 e 13 e 23 

with congeneric assumptions in which case the model is underidentified; how- 



ever, the unattenuated trait correlation p, 



m m 

1 2 i 3 
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I-Pt ij — c ( y 12 > y 25 ) c (y 2 2’ y i 3 ^ " ^( y p2* y 22 ) c ( y i 3 ^ y 23^ • Identification 
may be achieved with the congeneric model by repeating only one test (assum- 
ing Ajg = ) and using different method measures for Y 22 and y^^ in 

which case the model is: 



A = 



A 12 


0 


1 


0 


0 


0 




A 22 


0 


0 


1 


0 


0 




0 


A 13 


0 


0 


1 


0 




0 


A 23 


0 


0 


0 


1 


> 



(l6f) 



where A ^ ? = A^ by assumption, and 



0 = 



T 2 T 3 



0 



0 



0 



0 



'12 



0 



e 12 e i3 



0 



Symmetric 



'22 



0 



0 



'13 



'23 



J 



(l6g) 



This model is just identified (10 distinct elements in Z less 10 parameters 
to be estimated). Let us return to Cronbach's example in which there are 
Forms A (y^g) ® ( y 22 ^ initially and retests on Forms A ( y 2 j) and 

B ( y 2 ^ ) three years later. Cronbach partitions the variance using the 
immediate and retest correlations among forms (assumed parallel) which in our 
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model corresponds to the elements of Z • We may translate Cronbach’s parti- 
tioning procedure into functions of the model parameters in equations (l6a), 
(l6b), (l6c ), (l6d), and (l6e) as follows: 

1. "Lasting General Variance" = p(y^ 2 > y^,*,) = A *2 P ^ 2 ’ ^3^23 accor( ^ - 

ing to the model equals p(y 22 , y^) = A 2.2 p ( T 2’ T 3^ A *3 ' 

2. "Temporary General Variance" = p(y 12 , y 22 ) “ P( y i2’ y 23^ = ^12^22 ” 
A* 2 p(T 2 j Tg)A^ which according to the model equals p(y 12 > y 22 ) ~ 

p (y 22 > y i5 ) = A 12^22 ” A 22 P(T 2> T 3 )A 13 * In P rinci P le there is a 

different "Temporary General Variance" for the end time p(y^> yp^) - 

p(y 12 , y 25 ) = A * 3 A 23 - A 12 P ^ T 2 j t 3) a 23 which ep - uals p( y i3> ^23) ■ 

p (y 22 > y i3 ) = A 13 A 23 - A 22 p ( T 2> T 3^ A 13 • 

3. "Lasting Specific Variance" for Form A p(y-^ 2 > y-^) ” P( y l2 J y 23^ = 

p(y 12 > y 15 ) - p(y 2 2> y i3^ = A - (a£ 2 ) 2 p( e i2> e i3^ A - ^*3) 2 821(1 

for Form B p(y 22 , y 23^ “ P( y p 2 ’ y 23^ = p ( y 22’ y 23^ “ p ( y 22’ y l3^ = 

jl - (A* 2 ) p(e 22 , e 2 ^) Jl - (A*^) • 

4. "Temporary Specific Variance" [l - p(y^ 2 , y 22 )] " ^(^2’ y l3^ ” 

p(y 12 , y 25 )^ = [1 - P( y i2 J y 22^ " ^ p ( y 12 J y l 3 ^ ■ P^ y 22 J y l 3 ^ = 

1 " A 12 A 22 " A - ( A * 2 ) 2 P( e i2’ e 13^ A " ( A *3 ^ for the correlations 
used by Cronbach, but in principle there are three other temporary 

specific variances 1 - A^gA^g - Jl - (A* 2 ) 2 p(e 22 , e^) Jl - (A|^) , 

1 " A 13 A 23 “ /l " ^ A 12^ P( e i2 ; e 13^ v/ 1 ” ^ A 13^ ’ an<1 1 ” A 13 A 23 

Jl - (A* 2 f p(e 22 , e 2J ) Jl - (A * } f . 



ERiC 



25 




-2k- 






It can be seen that Cronbach 's procedure for partitioning of variance involves 
complicated functions of the model parameters. Wot only is it simpler to 
analyze observed correlations in terms of a set of structural parameters, but 
it allows for analysis of overidentified models. Further light can be shed 
on the assumptions implicit in the model of (l6a), (l6b), (l6c), (l6d), and 
(l6e) by asking what variables account for the correlated errors. Assuming 
that a single factor (M-^) underlies the correlation for Form A and another 
factor (Mg) for Form B the model becomes: 



where 



and 



y = p + AT + e* 



A 



y = (y 12 , y 2 2’ ^27) > 

T = (Tg, Ty M x , Mg) , 
e = (e^g, egg, e -^, e 23 ^ } 

1 0 B 12 0 ~ 

100 Bgg 

0 1 b i5 0 

010 b 25 , 



0 = 



Vm 

x 2 

Om m 

. 2 X 3 
0 

0 




1 

0 1 



(17a) 

(17b) 

(17c) 

(I7d) 



(17e) 



(17f) 
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Analysis of the identification problem shows that B-^j > ant ^ ®23 

are not separately identifiable; only the products (B^ ®13^ a 11 ^- (®22 ®23^ 
are identified. This means that in Joreskog's program we may arbitrarily 
set = B^, and B gg = Bg^ without disturbing the estimation for 

other parameters. Assuming B^g = B^ and B gg = B g^ , this model is a 
simple transformation of (l6a), (l6b), (l6c), (l6d), and (l6e) under essen- 
tially tau-equivalent assumptions, that is, V f V , V ^ V in 

e l2 e 22 e l3 e 23 

equation (l6e). In particular it can be seen that it must be assumed that 
and Mg are uncorrelated. It is possible to deal with oblique true and 
method factors but usually more different method measures are required as in 
our 3 trait x 3 method example in Section I. 

When methods of measuring a trait are made as different as possible, it 
is usually the case that the units of measurement are different, which means 
that congeneric rather than essentially tau-equivalent or parallel assump- 
tions are appropriate. Werts and Linn ( 1970a) consider growth models based 
on congeneric measures, e.g., in one case they use three congeneric measures 
of Tg and two congeneric measures of , allowing for same test correlated 
errors over time. This model is overidentified, but no attempt was made to 
deal with this complication. Phrasing this problem in terms of JiJreskog's 
general model: 

y = p + At + e (l8a) 

y = (yp2> ^ 22 } ^32* ^i3 j ^23^ (-^b) 

where y^g and y-j^ are linked as are ygg and y^ . 




T = (T 2 , T y M 1 , M 2 ) 



(18c) 
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13 
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(l8d) 



1 



$ 





0 0 1 
0 0 0 1 



(l8e) 



Assuming that A^g = A^, , Agg = Ag^ and for convenience that B^g = B^ , 

Bgg = Bg^ , this model has four overidentifying restrictions (15 distinct 

elements in Z less 11 parameters to be estimated). Werts and Linn give 

two formulas (1970a, p. 198, equations (28) and (29)) for estimating the 

correlation of status with gain involving observed correlations and variances 

whereas jBreskog's approach generates a single estimate by equation (15a). 

In essence Werts and Linn dealt with the elements of the observed variance- 

covariance matrix S which may yield inconsistent estimates of p . 

T 2 A 



whereas such inconsistency cannot arise with respect to the elements in Z . 
Jttreskog has an unpublished operating program for estimating factor scores 
within the confirmatory factor analysis model (jBreskog, 1971)* As Cronbach 
and Furby (1970) note, however, there is seldom need for such estimates. 
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Relationship to Factor Analysis 

A common practice in the factor analysis of growth data is to compare 

standardized factor loadings at one time to the loadings for the same set of 

measures at a later time. If the pattern of loadings remains constant over 

time the inference is drawn that the factors are measuring essentially the 

same dimension at different times. For example we might have three measures 

of Tg at time 1 with factor loadings A*g = .50 , = .40, and A^g = .50 

and identical loadings on when these measures are repeated at time 2, 

i.e.. A* - .50, A* = .40, and A* = .50 . For heuristic purposes let 

us suppose that the repetition of tests did not result in methods factors 

and that the true variance increased from V =1.0 to V_ = 1.5 over 

2 5 

time and C_ _ = 1.2 . It may be immediately inferred that the error vari- 

i 2 i 5 

ances for all tests increased over time since the test reliabilities (in 
this model the squared factor loadings ) remained constant and the true vari- 
ance increased. However, Wiley and Wiley (1970) have persuasively argued 
that it is more likely that error variances are a test characteristic which 
is likely to remain constant over time. If this is so, then an increase in 
true variance along the same dimension will necessarily mean that the relia- 
bilities of the tests will increase over time, i.e., the standardized factor 
loadings will increase. In the same fashion it may be deduced that if for 
any given test over time the unstandardized regression weights (A^g = A^*) 

and the error variances (V = V ) are equal, then in general the 

e i2 e i5 

standardized factor loadings (Af . ) are not proportional from one time to 
another. We conclude that comparison of standardized factor loading patterns 
over time provides no logical base for any conclusions about whether pretests 
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and posttests are measuring the same variable. It appears to us that such an 

assumption, which in this model is equivalent to equality of unstandardized 

regression weights over time (e.g., A-^g = A.^) , is basically not testable 

within the framework of this model. It would seem better not to make dubious 

assumptions that either the reliability or the error variance are relatively 

constant (over time) test characteristics, but to build models and gather 

requisite information such that these model parameters are identified. 

While it is not possible to test the assumption that g = A^ , it is 

quite possible for this assumption to be incompatible with the assumption 

that Agg = Ag^ . The ratio of to resulting from A^g = A^ may 

differ from the ratio resulting from Agg = Ag^ . This may be tested by the 
2 

increase in X (df = 1) resulting from the addition of A p g = A^^ to the 

model in which A^ p = A^ . Within the framework of this model, if it is 

true that the corresponding pairs of tests over time in fact have the same 

units, then the scaling of V„ to should be the same for each pair. 

i 3 X 2 

The finding that the data are consistent with the hypothesis that 



A, 0 = A, ^ and A, 
12 lp 



22 = Ag^ does not necessarily imply that the units of 



measurement for the corresponding pairs of tests over time are the same since 
it is quite possible for the scaling to be erroneous for both pairs of tests 
but in the same way. If the data are inconsistent with the hypothesis that 
A-j g = A.^, and A pp = A p ^ we could conclude that the units over time are not 
the same for both sets of tests, but it is still possible that the units are 
the same for one of the sets over time. Even if it could be shown that 
A^g = A^ , this would only be evidence consistent with, not proof of, the 
hypothesis that the scales are measuring the same process over time. 
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Determinants of Growth 



Werts and Linn (1970b) have considered the problem of making inferences 
about the determinants in a linear model. The Werts-Linn formulation was 
based on classical true score assumptions, i.e. , no provision was made for 
methods factors. For heuristic purposes let us reconsider the problem of 
growth determinants, formulating the three trait, three method model in 
terms of growth (T^ = T 2 + A) : 

T = (T 1 , T 2 , A, M 1 , M 2 , M 5 ) (19a) 
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i ’ 

It should, be noted that although this formulation does not directly involve 

the parameters of the underlying growth model A = ^ + P/yp + ^ ’ 

12 2 1 

however, the regression weights are: 



and 
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(I9d) 



(I9e) 



Traditional test theorists (e.g., Bloom, 19^4; Thorndike, 19^6) have been 
very concerned with and have drawn substantive inferences about the determinants 
of growth from the correlation of status with gain, usually corrected for 
"attenuation." However, as detailed by Werts and Linn ( 197 Oh), in a linear 
structural model prime interest is in the model parameters and 

D since if either one is zero the inference will be drawn that the 

at 2 .t x 



corresponding variable does not directly influence gain. Except in the case 
in which initial status is uncorrelated with all determinants of growth, 




r<2 



31 



knowledge of the correlation of status with gain, p ^ , does not allow us 

2 

to draw inferences about model parameters. It is quite possible for ^ 
to be completely spurious due to a common antecedent influence or it is quite 
possible for p m A to be zero without implying that D A „ or D.™ m 

be zero. For this reason we question Thorndike's (19 66 , p. 124) interpreta- 
tion: "In considerable part, the factors that produce gains during a 

specified time span appear to be different from those that produced the 
level of competence exhibited at the beginning of the period." Our objection 
is that Thorndike's conclusion was made from the correlation of status with 
gain, without specifically introducing into the analysis any presumed 
determinants of growth. In a linear structural model the total association 
of initial status with growth is an insufficient basis for drawing infer- 
ences about the various possible determinants of growth. 

Discussion 

The variety of test response tendencies covered by the rubric "methods 
factors" appear to be an almost universal complication in sociopsychological 
growth studies. Even though in principle the multitrait -multimethod model 
presented in this paper provides for "methods factors," it does not follow 
that this model does in fact provide a better simulation of reality than 
previous models which have typically ignored methods factors by assuming 
independent errors of measurement . It may be expected that our procedure 
will typically yield different parameter estimates (e.g., correlation of 
status with gain) than previous procedures, but what has been learned about 
growth and its determinants thereby? What is learned about reality from the 
overwhelming concern of the factor analyst with statistical fit? There is 
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no guarantee that the best fitting model yields substantively meaningful 
results (e.g., Werts, J&reskog, & Linn, in press). Why bother with complicated 
structural models involving unmeasured variables when it is likely that a 
simple regression equation invoicing only measured variables will provide the 
best prediction of the criterion? From our perspective, if the researcher's 
basic interest is in reality, then the research must be designed to explore 
reality, i.e., to offer evidence as to which of the initially plausible 
alternative hypotheses (models) provides the better simulation. In some 
cases this may involve a study of the theoretical implications to see what 
information is necessary to discriminate between the alternative models. In 
other cases the study may be a continuing one as in the building of models 
to simulate the national economy, in which case the ability to better predict 
new yearly data is used to discriminate among models. Our purpose in making 
these remarks is to heighten the awareness of researchers that parameter 
estimates, such as the reliability of gain scores, are always made within the 
framework of a whole set of untested assumptions about the nature of reality. 

It is misleading to talk about " the correlation of status with gain" since 
the meaning of this parameter is totally a function of the particular model 
used to derive the parameter. In most cases in which this type of estimate 
has been used, no effort has been made to examine the validity or even plausi- 
bility of the models underlying these estimates. The linear structural model 
presented herein is as suspect as any other model and needs to be justified 
as one of the plausible alternative hypotheses, prior to data analysis. 
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